Data Analysis From Scratch With Python: Step By Step Guide by Peters Morgan

Author:Peters Morgan [Morgan, Peters]
Language: eng
Format: epub, pdf
Publisher: AI Sciences LLC
Published: 2018-06-23T23:00:00+00:00

3.5, New York

2.0, California

6.7, Florida

If we use dummy variables, the above data will be transformed into this:

3.5, 1, 0, 0

2.0, 0, 1, 0

6.7, 0, 0, 1

Notice that the column for State became equivalent to 3 columns:

New York

California

Florida

3.5

1

0

0

2.0

0

1

0

6.7

0

0

1

As mentioned earlier, dummy variables indicate the presence or absence of something. They are commonly used as “substitute variables” so we can do a quantitative analysis on qualitative data. From the new table above we can quickly see that 3.5 is for New York (1 New York, 0 California, and 0 Florida). It’s a convenient way of representing categories into numeric values.

However, there’s this so-called “dummy variable trap” wherein there’s an extra variable that could have been removed because it can be predicted from the others. In our example above, notice that when the columns for New York and California are zero (0), automatically you’ll know it’s Florida. You can already know which State it is even with just the 2 variable.

Continuing with our work on 50_Startups.csv, we can avoid the dummy variable trap by including this in our code:

X = X[:, 1:]

Let’s review our work so far:

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.